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Foreword 


This  is  the  third  volume  of  a  technical  report  prepared  by  the  University  of  Dayton 
Research  Institute  for  the  Materials  and  Manufacturing  Directorate  (ML)  of  the  Air  Force 
Research  Laboratory  (AFRL),  Wright- Patterson  Air  Force  Base,  Ohio.  The  work  was 
performed  under  Contract  Number  F33615-95-C-5242  with  Mr.  Charles  F.  Buynak 
(AFRL/MLLP)  as  the  Air  Force  project  engineer.  The  technical  effort  was  performed 
between  29  September  1995  and  31  December  1999,  with  Dr.  Alan  P.  Berens  of  the 
University  of  Dayton  Research  Institute  as  the  principal  investigator. 

The  final  report  of  this  work  comprises  three  volumes.  Volume  1  presents  a  description 
of  changes  made  to  the  probability  of  detection  (POD)  analysis  program  of  Mil-STD- 
1823  and  the  statistical  evaluation  of  modifications  that  were  made  to  version  3  of  the 
Eddy  Current  Inspection  System  (ECIS  v3).  Volume  2  contains  the  Users  Manual  for  the 
version  3  update  of  the  POD  program.  The  results  of  a  separate  study  for  predicting  POD 
from  specimens  of  like  geometry  and  materials  are  presented  in  Volume  3. 
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Section  1 

Introduction 


In  the  current  environment  of  restricted  budgets,  aircraft  component  parts  are  often  used  beyond 
their  design  life.  In  order  to  avoid  the  large  cost  of  replacing  critical  rotating  parts  as  they  reach 
their  “safe-life”  limits,  a  Retirement  For  Cause  (RFC)  program  was  implemented.  The  RFC 
program  involves  periodic  nondestructive  evaluation  (NDE)  inspections  to  assess  the  damage-state 
of  components.  Components  with  no  detectable  cracks  are  returned  to  service  and  those  with 
detected  cracks  are  discarded  before  they  can  cause  an  incident.  This  process  allows  parts  with 
high  life  to  be  used  to  their  full  potential.  The  Air  Force  has  embraced  RFC  and  currently  uses  it 
successfully  to  manage  component  life  for  several  of  their  gas  turbine  engines. 

Managing  the  structural  integrity  of  engine  parts  through  the  RFC  program  requires  quantifying  the 
capability  of  the  NDE  system  used  in  the  inspections.  The  RFC  uses  the  highly  automated  Eddy 
Current  Inspection  System  (ECIS)  developed  by  Veridian.  The  capability  of  the  RFC/ECIS  has 
been  quantitatively  evaluated  by  inspecting  specimen  sets  of  representative  material  by  geometry 
combinations.  The  ECIS  response  (d)  at  cracks  of  known  size  {a)  are  then  analyzed  to  estimate 
the  probability  of  detection  (POD)  as  a  function  of  crack  size,  or  POD(a).  The  reliably  detected 
crack  size,  ago,  defined  as  the  crack  size  for  which  POD(a9o)  =  0.90,  is  used  as  the  one  number 
characterization  of  the  ability  of  ECIS  to  detect  cracks  in  the  material  by  geometry  combination. 

The  concept  of  obtaining  POD(a)  from  d  versus  a  data  is  shown  in  Figure  1.  The  probability  of 
detection  of  a  random  crack  of  some  fixed  size  is  determined  from  the  scatter  in  the  NDE  system 
response  about  the  mean  response  for  cracks  of  that  given  size  [1].  Experience  has  shown  that  a 
linear  fit  of  log  d  versus  log  a  is  often  a  reasonable  model  for  the  mean  response  over  some  range  of 
crack  sizes.  In  a  range  of  crack  sizes  for  which  the  linear  fit  is  reasonable  and  the  scatter  about  the 
fit  is  normally  distributed  and  does  not  depend  on  crack  size,  POD(fl)  will  be  a  cumulative  normal 
distribution  function  with  //  and  eras  given  by  the  following  equations: 


(1) 
(2) 

(3) 

and  =  the  difference  in  response,  di,  of  crack  i  from  the  mean  of  all  cracks  of  size  a; 
s  =  is  the  standard  deviation  of  ei; 

ddec  =  is  the  response  decision  threshold  for  a  crack  indication; 

Bo„  B],  and  5'  are  fit  parameters  that  are  estimated  from  the  data. 

The  ago  value  is  then  given  by  the  equation; 


where 


//  =  (log  ddec  -  Bo)/Bj 
(7=  s/Bj 

log  di  =  Bo  +  Bj*  log  ai  +  eu 


ago  =  exp  (|i  -I- 1.282*0). 


(4) 
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POD  calculated  from  three  parameters: 

Intercept  of  line  -  Bg 
Slope  of  line  - 

Standard  deviation  of  a  deviations  from  line  -  s 
Parameters  of  POD  model: 

/'  =  (In  Sthr  ■  ®0  )^®1 

CT  =  s/B^  o  o 


SIGNAL  THRESHOLD 
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Crack  Depth  (mil) 
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Figure  1.  Schematic  for  Obtaining  PODfaj  from  d  versus  a  Data 


To  date,  appropriate  material/geometry  eombinations  have  been  used  to  simulate  real  inspections 
for  the  POD(a)  charaeterization  of  RFC/ECIS  capability,  and  the  three  parameters  of  the  d  versus 
a  analysis  are  estimated  directly  from  the  inspeetion  results.  To  avoid  the  necessity  of  produeing 
specimen  sets  for  all  materials  by  geometry  combinations  that  will  be  inspeeted,  a  method  for 
eorrelating  POD  results  between  speeimen  sets  is  being  sought.  Two  eharaeterization  concepts 
to  account  for  the  differenees  in  geometry  by  material  combinations  have  been  suggested.  These 
eoneepts  attempt  to  predict  the  unknown  parameters  of  the  log  d  versus  log  a  fit  for  one  geometry 
of  one  material,  from  inspeetions  of  three  similar  speeimen  sets,  (1)  the  same  geometry  of  a 
second  material,  (2)  flat  plate  data  of  the  same  material,  and  (3)  flat  plate  data  on  the  second 
material.  Each  concept  attempts  to  predict  Bo,  Bi,  and  5',  the  unknown  parameters  of  the  log  d 
versus  log  a  fit,  from  values  obtained  from  inspeetions  of  the  other  three  speeimen  sets.  Beeause 
these  methods  involve  material-based  adjustments  to  the  parameters  obtained  from  the  same 
geometry,  these  methods  are  also  referred  to  as  material  eorrelation  methods.  The  two  material 
eorrelation  methods  are  referred  to  as  the  “difference”  and  “ratio”  methods,  sinee  the  parameters 
of  the  missing  eombination  are  obtained  either  from  differences  or  ratios  of  the  parameters  from 
the  available  sets. 

To  further  demonstrate  the  eoneept  of  material  correlation.  Table  1  represents  example 
eomponents  with  five  types  of  materials  and  five  types  of  geometries.  Parameters  for  material 
and  geometry  combinations  in  the  first  row  and  first  eolumn  are  assumed  to  be  known. 
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Parameters  for  all  other  material  and  geometry  combinations  must  be  inferred  from  the  material 
correlation  method. 


Material 


Table  1.  Geometry  -  Material  Combinations 


Geometry 


A  material  correlation  study  performed  by  Veridian  in  September  1999  [2]  was  motivated  by  the 
need  to  minimize  the  required  number  of  specimen  sets  while  providing  the  most  accurate 
inspections  possible.  The  primary  proposal  of  the  material  correlation  project  was  that  the  ratio 
of  the  eddy  current  responses  from  cracks  of  the  same  size  in  two  different  materials  of  the  same 
geometry  should  equal  the  ratio  of  the  eddy  current  responses  from  cracks  in  specimens  made 
from  the  same  two  materials  having  a  different  geometry.  Because  the  d  versus  a  data  is 
constructed  in  a  log-log  domain,  the  proposed  materials  correlation  assumption  equates  to 
parameter  differences. 

The  goal  of  the  study  was  to  show  whether  or  not  reliability  analysis  results  obtained  from  the 
eddy  current  inspection  of  certain  specimen  sets  could  be  used  to  predict  the  reliability  analysis 
results  for  another  specimen  set  of  like  material  but  with  different  geometries.  For  example,  can 
the  reliability  analysis  parameters  be  determined  for  Ti-6246  bolt  hole  specimens  using  bolt  holes 
in  IN  718  and  flat  plate  specimens  of  IN  718  and  Ti-6246?  To  test  this,  the  predicted  probability 
of  detection  was  compared  to  the  actual  POD  using  combinations  of  materials  and  geometries 
that  already  exist,  as  shown  in  Table  2. 


Table  2.  Material  Correlation  Test  Matrix 


Geometry 

Material 

1 

0.342"  Bolt  Hole 

IN718 

2 

0.460"  Bolt  Hole 

Ti-6246 

3 

Flat  Plate 

Ti-6246 

4 

Flat  Plate 

IN718 

3 


Researchers  at  the  University  of  Da3^on  Research  Institute  (UDRI)  reviewed,  analyzed,  and 
evaluated  the  study  to  predict  POD  through  the  material  correlation  study  performed  by  Veridian. 
This  report  will  present  a  discussion  of  requirements  for  performing  materials  correlation  in 
Section  2,  examination  of  the  data  acquisition  methods  of  the  study  in  Section  3,  and  an 
evaluation  of  the  POD  inference  methods  using  the  data  collected  in  the  study  in  Section  4. 
Finally,  general  recommendations  for  optimizing  the  material  correlation  study  will  be  provided 
in  Section  5,  and  Section  6  summarizes  the  study  and  its  results. 
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Section  2 

Requirement  for  Performing  Material  Correlation  Studies 


Many  variables  must  be  controlled  to  acquire  eddy  current  responses  from  different  sets  of  test 
specimens  if  they  are  to  be  useful  for  material  correlation  studies.  Some  of  the  known  variables 
that  can  affect  the  amplitude  of  the  eddy  current  responses  from  a  crack  are  listed  in  Table  3. 


Table  3.  Eddy  Current  Inspection  Parameters 


Category 

Variable 

Controllable 

Probe 

Inherent  Coil  Sensitivity 

Relative/Differential  Sensitivity 

y 

Coil  size 

y 

Lift-off 

y 

Instrument 

Gain 

y 

Phase  Angle 

y 

Frequency 

y 

Filters 

y 

Specimen 

Conductive 

y 

Crack  Orientation 

Crack  Opening 

Aspect  Ratio  Depth 

Residual  Stress 

Grain  Size 

Inspection 

Step  Size 

y 

Calibration  Variation 

Scan  Speed 

y 

Material  Noise 

y 

Random  Noise 

y 

Geometry 

Lift-off 

y 

Geometric  Variables 

y 

An  extensive  amount  of  work  is  required  to  examine  each  variable  and  determine  the  extent  of 
how  and  why  it  affects  the  eddy  current  response  data  acquired  from  cracks  in  reliability  specimens. 
It  would  be  difficult,  if  not  impossible,  to  control  all  of  the  variables.  If  they  are  not  controlled, 
the  effect  of  not  controlling  them  must  be  understood  and  predicted.  This  is  the  crux  of  the 
problem.  Since  many  of  these  variables  necessitate  different  material/geometry  combinations,  they 
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cannot  be  strictly  controlled.  Therefore,  they  must  be  understood  and  their  behavior  predicted.  The 
question  that  remains  unanswered  is:  How  well  can  this  be  done? 

Phase  angle  calibration  is  a  good  example  of  how  difficult  it  is  to  manage  a  variable  during  eddy 
current  inspection.  Differential  eddy  current  probes  are  commonly  used  for  RFC-type  inspections 
to  achieve  the  signal-to-noise  ratios  needed  to  detect  small  cracks.  The  differential  nature  of  the 
probes  greatly  affects  the  phase  angle  setting  in  the  eddy  current  instrument.  All  responses  from  the 
probe  result  from  the  difference  between  the  signals  coming  from  the  two  coils.  Thus,  the  phase 
angle  setting  is  as  much  the  result  of  the  imbalance  of  the  two  coils  as  it  is  the  interaction  of  the 
eddy  currents  with  the  electrical  properties  of  the  material.  The  responses  from  each  coil  are 
dependent  on  the  material,  the  crack,  and  the  relationship  between  each  coil  and  the  metal  in  its 
proximity.  It  is  difficult  to  determine  how  each  differential  probe  responds  to  the  unique 
combination  of  its  own  imbalance,  the  specimen,  geometry  type,  and  the  crack.  Acquiring  eddy 
current  responses  from  differential  probes  from  three  specimen  sets,  and  then  using  the  data  to 
predict  the  eddy  current  response  for  cracks  in  a  fourth  specimen  set,  requires  a  thorough 
understanding  of  the  probe’s  interaction  with  the  fourth  specimen’s  material  and  geometry,  as  well 
as  the  cracks. 

Essentially,  the  problem  in  this  example  is  twofold: 

1 .  Whether  the  unknown  calibration  phase  angle  could  be  predicted  for  a  material/geometry 
combination  based  on  the  known  phase  angles  of  three  other  material/geometry  combinations 

2.  Given  that  unknown  calibration  phase  angles  can  be  predicted  and  the  phase  angle  can  be  set 
accordingly,  whether  the  crack  response  (phase  and  amplitude)  could  be  predicted  for  this 
material/geometry  combination  from  the  crack  response  on  different  cracks  found  in  the  other 
three  material/geometry  combinations. 

These  two  problems  must  be  resolved  in  light  of  the  other  variables,  such  as  lift-off.  The 
question  deals  with  modeling  the  phase  response  and  then  the  crack  response  behavior  at  that 
phase  angle  setting. 

Another  issue  that  complicates  any  effort  to  do  materials  correlation  is  the  application  of  the 
materials  correlation  results  to  real  engine  part  inspections.  The  conditions  under  which  the 
materials  correlation  tests  are  conducted  must  be  reproduced  in  the  inspection  of  real  engine 
parts.  Such  inspection  conditions  include  coil  type,  inspection  algorithm,  calibration  (phase  and 
gain  calibration),  scan  rates,  filter  settings,  inspection  parameter  values,  and  others.  Often, 
however,  the  particulars  of  a  real  engine  part  inspection  will  constrain  these  variables.  This  means 
that  the  materials  correlation  tests  must  be  designed  with  the  inspection  in  mind. 

An  example  of  this  would  be  bolt  hole  inspection  of  a  material  for  which  there  is  no  POD 
specimen  set.  These  inspections  are  conducted  with  noncontact  bolt  hole  probes  scanning  at  a 
very  high  rate.  Many  bolt  hole  inspections  are  conducted  at  1500  RPM.  A  bolt  hole  diameter  of 
0.5  inches  gives  a  surface  velocity  of  about  40  inches  per  second.  Performing  a  materials 
correlation  using  flat  plate  and  bolt  hole  specimens  of  the  appropriate  materials  would  require 
attempting  to  collect  flat  plate  data  using  proper  controls.  However,  it  would  be  virtually 
impossible  to  match  surface  velocity  on  the  flat  plate  specimens  because  the  RFC  linear  axes 
have  a  maximum  velocity  of  300  inches  per  minute,  or  5  inches  per  second.  The  flat  plate 


6 


specimens  could  be  mounted  in  a  test  fixture  clamped  to  the  turntable,  which  is  routinely  done, 
and  the  turntable  rotated.  The  current  flat  plate  fixture  holds  specimens  at  a  radius  of  about  7 
inches.  At  the  maximum  turntable  speed  of  18  revolutions  per  minute,  the  surface  velocity 
would  be  only  14  inches  per  second.  Since  it  is  apparent  that  the  surface  velocity  cannot  be 
matched,  the  only  alternative  is  to  assume  that  the  slower  surface  velocity  can  be  accounted  for 
by  scaling  all  parameters  that  are  scan-speed  dependent,  such  as  filter  settings  and  inspection 
parameters. 

Another  variable  that  would  need  to  be  controlled  in  this  example  is  lift-off.  Since  bolt  hole 
probes  are  noncontact,  the  flat  plate  specimens  would  have  to  be  scanned  with  the  same  lift-off, 
which  poses  implementation  challenges  for  a  surface  probe.  Another  issue  is  the  type  of  notch  used 
to  calibrate  the  probe.  Bolt  hole  probes  are  calibrated  using  corner  notches,  and  surface  probes 
are  calibrated  using  surface  notches.  The  surface  probes  on  the  corner  notches  used  for  bolt  hole 
calibration  would  have  to  be  calibrated  to  remove  this  source  of  variation.  Since  this  is 
impossible,  an  alternative  would  be  to  use  the  bolt  hole  probe  to  perform  the  flat  plate  specimen 
inspection.  Obviously,  this  would  be  difficult  if  not  impossible,  especially  in  light  of  the  need  to 
keep  lift-off  uniform;  however,  it  has  the  advantage  of  removing  the  coil  as  a  variable. 

Another  complication  in  this  example  is  the  need  to  control  the  inspection  routine.  The  bolt  hole 
inspection  routine,  including  inspection  parameters,  step  sizes,  and  sampling  rates,  would  have  to 
be  used  on  the  flat  plate  inspection.  If  any  of  these  steps  cannot  be  controlled,  then  the  results  of 
the  tests  and  models  would  depend  upon  the  validity  of  the  assumptions  used  to  ignore  the 
differences. 

It  may  be  impractical,  if  not  impossible,  to  control  the  variables.  In  Veridian’s  attempt  to  test  a 
proposed  model  for  material  correlation,  an  application  was  selected  that  would  allow  many  of 
the  variables  to  be  controlled.  Their  tests  and  controls  will  be  discussed  in  more  detail  in  the 
next  section.  While  Veridian  used  flat  plate  and  bolt  hole  specimens,  they  tested  the  materials 
correlation  model  using  a  scallop  setup  and  inspection  method.  They  had  to  design  a  special 
probe  that  could  fit  into  a  bolt  hole  and  also  ride  on  a  surface  specimen.  They  used  a  special 
phase  and  gain  calibration  routine.  In  effect,  Veridian  created  a  hybrid  test  condition  to  avoid  the 
problems  that  would  be  encountered  in  an  attempt  to  apply  materials  correlation  to  a  real 
inspection.  This  is  not  a  criticism  of  Veridian’s  methods,  but  a  concession  to  the  difficulty  of 
controlling  the  variables. 

From  this  discussion,  it  is  clear  that  each  inspection  application  forces  a  unique  set  of  testing 
conditions  on  the  material  correlation  test  setup.  The  uniqueness  of  the  test  conditions  almost 
assures  its  inapplicability  of  results  to  other  applications.  This  narrow  applicability  results  in  the 
requirement  of  three  sets  of  data  for  each  and  every  possible  material/geometry  combination  for 
which  there  are  no  specimen  sets.  Materials  correlation  trades  the  cost  of  one  set  of  new 
specimens  for  the  cost  of  three  POD  runs  of  data  on  existing  specimens,  plus  the  engineering  cost 
of  designing  and  creating  the  scan  plans  and  possibly  other  hardware  (such  as  probes)  to  carry  out 
these  runs.  This  assumes  that  the  required  test  conditions  can  be  met. 
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One  final  complicating  factor  is  that  not  all  geometries  have  a  corresponding  POD  specimen  set 
in  any  material.  This  case  indicates  the  existence  of  what  might  be  called  a  geometry  correlation; 
that  is,  POD  is  determined  on  an  existing  set  of  specimens  of  one  geometry  and  it  is  assumed  that 
these  results  apply  to  another  geometry.  For  instance,  a  scallop  set  of  specimens  might  be  used  to 
represent  an  anti-rotation  tab  fillet  radius.  Issues  of  probe,  coil,  lift-off,  and  curvature  further 
complicate  the  assumptions  made  for  materials  correlation. 
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Section  3 

Examination  of  Data  Acquisition 


As  mentioned  in  the  previous  section,  Veridian  created  a  hybrid  test  condition,  which  was 
essential  for  testing  materials  correlation.  Veridian  understood  this,  as  shown  by  the  following 
quote  from  their  report:  “A  primary  concern  in  this  project  was  to  attempt  to  minimize  any 
variation  in  the  multitude  of  parameters  in  data  collection,  including,  among  others,  gain,  phase, 
lift-off,  coil  orientation,  filtering,  scan  speed  and  signal  processing”.  As  mentioned  earlier, 
Veridian  designed  and  built  a  special  probe  that  would  allow  them  to  scan  both  the  bolt  hole  and 
flat  plate  specimens.  The  probe  could  be  configured  in  two  different  ways  to  further 
accommodate  this  constraint.  Veridian  also  created  special  software  to  allow  the  routine  to 
perform  inspections  on  both  specimen  sets.  Necessity  dictates  differences  in  setup  for  these 
different  inspections,  but  as  noted  by  Veridian,  “Critical  functions  were  performed  by  common 
subroutines.”  Inspection  setup  was  chosen  to  be  consistent  with  the  scallop  inspection. 

To  perform  gain  calibration,  Veridian  used  the  master  set  of  reference  standards  to  avoid 
additional  variation  due  to  notch  parameter  uncertainties.  They  performed  phase  calibration  on 
the  IN  100  reference  block  across  the  long  notches.  The  notch  signal  was  then  rotated  into  the 
vertical  component  of  the  impedance  plane.  For  Ti-6246  materials,  an  additional  phase  calibration 
was  performed.  Gain  calibration  for  both  materials  was  performed  on  the  IN  100  block  after  phase 
calibration  on  the  IN  100  notch.  For  Ti-6246  specimens,  the  phase  was  rotated  after  gain 
calibration  to  be  consistent  with  the  phase  determined  on  the  Ti-6246  notch.  This  strategy  was 
used  to  reduce  variation  caused  by  gain  calibration  -  Veridian  stated,  “This  method  of  calibration 
was  to  provide  consistent  gain  setup  across  all  data  sets  while  compensating  for  phase  rotation  in 
dissimilar  metals”  [2].  The  propriety  of  such  a  protocol  is  arguable,  but  their  attempt  further 
emphasizes  the  difficulty  in  performing  a  good  material  correlation. 

Veridian  describes  their  method  of  selecting  which  cracks  to  use  on  their  test.  The  need  for  this 
selection  process  arose  out  of  the  design  of  the  specimen  set.  Since  the  specimen  set  was 
designed  to  accommodate  fluorescent  penetrant  inspections,  there  were  multiple  cracks  on  any 
given  specimen,  which  could  be  in  different  orientations.  Veridian  eliminated  those  cracks  that 
were  inappropriate  for  this  test. 

Veridian  used  a  D20  differential  coil  scanned  at  60  inches  per  minute  (1  inch  per  second),  with  a 
sampling  rate  of  1000  points  per  second.  The  high  pass  filter  was  set  at  5  Hz,  and  the  low  pass 
filter  at  100  Hz.  For  a  D20  coil  scanning  at  this  rate,  a  typical  flaw  frequency  would  be  in  the 
range  of  30  to  60  Hz.  These  filter  settings  appear  acceptable  for  this  configuration.  Other 
parameters  were  not  identified  in  their  report,  but  there  is  reason  to  believe  that  the  parameters 
are  set  correctly  and  properly  controlled,  especially  in  light  of  the  care  demonstrated  in  the  rest  of 
the  setup  to  maintain  strict  control.  Overall,  Veridian  took  great  care  to  control  the  conditions 
under  which  data  is  collected  for  this  test.  These  ideal  circumstances  should  be  adequate  to  test 
the  hypotheses  inherent  in  material  correlation. 
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Section  4 

Evaluation  of  POD  Inference  Methods  with  Test  Data 


As  stated  earlier,  two  methods  have  been  proposed  for  inferring  the  POD(a)  function  from 
d  versus  a  data:  the  difference  method  and  the  ratio  method.  The  methods  are  described  and 
applied  to  the  data  collected  on  ECIS  inspections  of  IN  718  and  Ti-6246  bolt  hole  and  flat  plate 
specimen  sets. 

4.1  Proposed  Correlation  Methods  for  Inferring  POD 

Veridian  designed  and  conducted  a  special  experiment  using  bolt  hole  specimens  to  validate  or 
refute  the  proposed  material  correlation  methods.  The  ECIS  system  was  used  to  inspect  four 
specimen  sets:  IN  718  bolt  holes  and  flat  plates,  and  Ti-6246  bolt  holes  and  flat  plates.  The 
inspections  were  conducted  to  provide  data  that  would  mimic  the  bolt  hole  inspections  as  closely  as 
possible.  The  data  analysis  to  evaluate  the  two  methods  assumed  that  the  objective  was  to  estimate 
the  POD(a)  capability  for  IN  718  bolt  hole  inspections  under  these  test  conditions  from  the  known 
log  d  vs  log  a  parameters  of  the  other  three  combinations  of  geometry  and  material.  Calculations  of 
the  POD  parameters  for  the  three  known  combinations  are  shown  in  the  following  equations,  where 
TiFP  =  Titanium  Plat  Plate,  TiBH  =  Titanium  Bolt  Hole,  InFP  =  IN  100  Plat  Plate,  and  InBH  =  IN 
100  Bolt  Hole. 


Por  the  Ti-6246  flat  plates,  let 

log  d(TiFP)  =  Bo(TiFP)+  Bi(TiFP)*\og  a  +  e(TiFP),  (5) 

where  e  (TiFP)  is  N(0,5  (TiFP)).  Por  the  IN  718  flat  plates,  let 

log  d(INFP)  =  Bo(INFP)+  Bi(INFP)*\og  a  +  e(INFP),  (6) 

where  e  (INFP)  is  N(0,5  (INFP)).  Por  the  Ti-6246  bolt  holes,  let 

log  d(TiBH)  =  Bo(TiBH)+  Bi(TiBH)nog  a  -l-  e(TiBH),  (7) 

where  e  (TiBH)  is  N(0,5'  (TiBH)).  The  objective  is  to  estimate  the  parameters  of  the  fit  that  would 
be  obtained  if  the  IN  718  bolt  hole  specimens  were  available  for  inspection.  That  is,  the  objective  is 
to  estimate  BO(INBH),  BI(INBH),  and  5  (TiBH). 

4.1.1  Difference  Method 

The  difference  method  for  estimating  the  IN  718  bolt  hole  parameters  is  partially  related  to  the 
assumption  that  the  signal  response  ratio  between  the  flat  plate  specimens  of  two  different 
materials  would  be  the  same  as  the  signal  response  ratio  between  bolt  holes  of  the  same  two 
different  materials.  Under  this  assumption,  the  following  equation  applies: 


d(INBH)/d(TiBH)  =  d(INFP)/d(TiFP). 


(8) 
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Then, 


log  d(INBH)  -  log  d(TiBH)  =  log  d(INFP)  -  log  d(TiFP).  (9) 


Substituting  equations  (5)  through  (7),  rearranging  terms  and  equating  coefficients  yields  the 
following  equations: 


Bo(INBH)  =  BoiTiBH)  +  [Bo(INFP)  -  Bo(TiFP)],  (10) 

Bi(INBH)  =  Bi(TiBH)  +[Bi(INFP)  -  Bi(TiFP)],  (1 1) 

and  e  (INBH)  =  e  (TiBH)  +[e  (INFP)  -  e  (TiFP)].  (12) 

The  calculations  of  the  slope  and  intercept  of  the  log  d  versus  log  a  fit  for  the  IN  718  bolt  holes  is 
straightforward  from  equation  (8)  and  are  given  in  equations  (10)  and  (1 1).  The  assumed  physical 
basis  for  assigning  some  validity  to  the  difference  method  is  that  the  slope  and  intercept  of  the  fit 
are  obtained  from  the  slope  and  intercept  of  the  bolt  holes  of  the  other  material  and  then  adjusted 
by  a  material  factor  that  is  calculated  from  the  flat  plates,  i.e.,  a  different  geometry.  However,  the 
physical  basis  justifying  the  estimation  of  the  parameter  s  (INBH)  is  not  understood. 


In  the  d  analysis,  5'  (INBH)  is  the  standard  deviation  of  the  normal  distribution  of  the  residuals,  e 
(INBH),  of  the  d  values  from  the  linear  log-log  fit  implied  by  equation  (3).  Equation  (12)  cannot 
be  used  to  calculate  5'  (INBH)  because  the  co-variances  of  the  residuals  from  the  fit  of  the  other 
materials  by  geometry  combinations  are  not  known.  In  the  proposed  difference  approach,  the 
somewhat  arbitrary  assumption  is  made  that  the  difference  between  the  residual  standard 
deviations  of  the  IN  718  and  Ti-6246  bolt  holes  would  equal  the  difference  between  the  residual 
standard  deviations  of  the  IN  718  and  Ti-6246  flat  plates.  This  assumption  results  in  the  formula 
shown  in  equation  (13).  Note  that  equation  (13)  is  the  result  of  an  arbitrary  assumption  that 
would  not  be  valid  for  all  applications.  Reasonable  data  could  be  postulated  that  would  lead  to 
the  impossible  situation  of  a  negative  standard  deviation. 


5  (INBH)  =  5  (TiBH)  +[s(INFP)  -  s  (TiFP)] 


(13) 


Equations  (10),  (1 1),  and  (13)  summarize  the  difference  method  proposed  for  predicting  the  d 
versus  a  parameters  for  the  missing  specimen  set.  These  parameters  were  proposed  for  use  in 
equations  (1),  (2),  and  (4)  to  characterize  the  POD(a)  function  and  the  apo  value  for  selected  dthr 
values. 


4.1.2  Ratio  Method 


The  concept  of  the  ratio  method  is  somewhat  similar  to  that  of  the  difference  method,  with  the 
exception  that  the  adjustments  for  material  differences  are  made  in  terms  of  the  ratios  of 
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parameters  rather  than  the  differences.  No  justification  for  these  somewhat  arbitrary  assumptions, 
in  terms  of  differences  in  signal  responses,  is  made.  In  this  method,  it  is  simply  assumed  that: 


Bo(INBH)  =  BoiTiBH)  *  [Bo(INFP) /Bo(TiFP)],  (14) 

Bi(INBH)  =  Bi(TiBH)  *[Bi(INFP) /Bj(TiFP)],  (15) 

and  5  (INBH)  =  s  (TiBH)  *[s  (INFP)  /  s  (TiFP)].  (16) 


The  ratio  method  will  not  result  in  impossible  estimates  for  the  missing  parameters,  but  large 
biases  can  be  introduced  if  there  are  significant  differences  in  the  slope  or  standard  deviation 
estimates  due  to  a  specific  specimen  set. 

4.2  Application  of  the  Methods 

Veridian  collected  four  sets  of  data  that  can  be  used  to  evaluate  any  proposed  method  for 
inferring  POD(fl)  for  a  missing  specimen  set.  The  objective  for  collecting  these  experimental 
data  was  to  provide  a  database  that  would  permit  estimating  the  POD(fl)  parameters  from  the 
proposed  methods  while,  at  the  same  time,  having  a  data  set  that  represented  a  true  set  of  d 
versus  a  data  from  the  missing  geometry-material  specimens.  The  inspection  data  were  from  IN 
718  and  Ti-6246  bolt  hole  and  flat  plate  specimen  sets  and  were  obtained  under  carefully 
controlled  conditions  that  were  similar  to  a  standard  bolt  hole  inspection.  Plots  of  the  four  sets  of 
d  versus  a  data  are  presented  as  Figures  2  through  5.  Parameters  of  the  fit  for  each  specimen  set 
were  obtained  for  the  widest  crack  size  range  in  which  the  required  statistical  assumptions  were 
reasonably  valid.  The  range  of  crack  depth  validity  for  these  POD(fl)  analyses  was  approximately  8 
to  22  mil.  The  estimates  of  the  parameters  of  the  fits  to  these  data  are  presented  in  Table  4. 


Table  4.  Fits  of  log  d  versus  log  a  from  Inspection  Data 


IN  718  BH 

Ti-6246  BH 

IN  718  FP 

Ti-6246  FP 

Flaw  Depth  Range: 

8-21 

8-25 

8-24 

7.8-19.7 

Cracks  Analyzed: 

35 

40 

21 

28 

Signal  Minimum: 

500 

500 

200 

200 

Signal  Maximum: 

8192 

8192 

8192 

8192 

Parameter 

Parameter  estimate  from  inspection  data 

Intercept  (BO) 

5.16 

4.04 

3.76 

4.26 

Slope  (Bl) 

1.12 

1.53 

1.52 

1.40 

Residual  Error 

0.201 

0.162 

0.180 

0.152 

Repeatability  Error 

0.058 

0.032 

0.119 

0.036 

Sigma 

0.183 

0.107 

0.131 

0.110 

The  parameter  estimates  from  the  flat  plate  specimens  and  one  of  the  bolt  hole  specimen  sets 
were  then  used  to  estimate  the  parameters  of  the  missing  bolt  hole  inspection  using  both  the 
difference  and  ratio  inference  methods. 
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4.3  Data  Analysis 

The  comparison  of  the  direct  and  inferred  POD(a)  results  from  the  two  methods  are  presented  in 
the  following  subsections. 

4.3.1  Difference  Method 

The  difference  method  parameter  estimates  of  the  log  d  versus  log  a  fit  for  the  missing  bolt  hole 
specimens  are  presented  in  Table  5.  Also  repeated  in  Table  5  are  the  values  obtained  from  analysis 
of  the  actual  data.  The  difference  method  would  underestimate  the  intercept  and  sigma  and 
overestimate  the  slope  when  the  IN  718  bolt  holes  were  the  unknown  set.  The  reverse  was  true 
when  the  Ti-6246  bolt  hole  parameters  were  estimated.  In  this  example,  the  difference  correction 
for  materials  that  is  obtained  from  the  flat  plate  specimens  degrades  the  fit  that  would  be  obtained 
simply  by  ignoring  material  differences.  The  intercept  and  slope  parameters  from  the  other  material 
are  closer  to  the  actual  values  than  are  those  obtained  by  the  additional  “material”  adjustment. 

Table  5.  Comparison  of  log  d  versus  log  a  Parameter  Estimates  from  the  Difference  Method 


IN  718  BH 

Ti-6246 

Parameter 

Data 

Inferred 

Data 

Inferred 

Intercept  (BO) 

5.16 

3.54 

4.04 

5.66 

Slope  (Bl) 

1.12 

1.66 

1.53 

1.00 

Sigma 

0.183 

0.127 

0.107 

0.162 

To  see  the  effect  of  the  differences  in  parameter  estimates  on  the  characterization  of  inspection 
capability  in  terms  of  ago  values,  a  plot  of  ago  versus  athr  was  generated  for  the  approximate  range  of 
validity  of  the  fits.  These  comparisons  are  presented  in  Figures  6  and  7  for  the  IN  718  and  Ti-6246 
bolt  hole  specimens,  respectively.  While  it  is  not  unusual  for  ago  values  to  differ  by  about  0.001 
inch  (1  mil)  due  to  changes  in  probes  and  recalibrations,  the  differences  of  3  to  4  mil  at  the  smaller 
crack  sizes  are  unacceptable.  This  is  particularly  true  for  the  inferred  Ti-6246  thresholds,  since  the 
ago  values  for  an  inferred  threshold  would  not  be  conservative.  In  a  blind  application  of  the  method, 
there  would  be  no  way  of  knowing  whether  or  not  the  results  were  not  conservative. 

4.3.2  Ratio  Method 

The  ratio  estimates  for  the  parameters  of  log  d  versus  log  a  fit  for  the  missing  bolt  hole 
specimens  are  presented  in  Table  6.  Also  repeated  in  Table  6  are  the  values  obtained  from  the 
data.  The  results  are  analogous  to  those  obtained  from  the  difference  method.  Again,  the  ratio 
correction  for  material  obtained  from  the  flat  plate  specimens  degrades  the  fit  that  would  be 
obtained  simply  by  ignoring  the  material  difference  adjustment  from  the  flat  plates. 

The  plots  of  ago  versus  athr  for  characterizing  the  IN  718  and  Ti-6246  capability  as  a  function  of 
decision  threshold  using  the  ratio  method  are  presented  in  Figures  8  and  9.  Again  the  differences 
that  would  result  from  the  inferred  ago  are  judged  to  be  too  large.  As  with  the  difference  method, 
the  inferred  ago  value  for  the  Ti-6246  bolt  holes  are  not  conservative. 
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Table  6.  Comparison  of  log  d  versus  log  a  Parameter  Estimates  from  the  Ratio  Method 


IN  718  BH 

Ti-6246 

Parameter 

Data 

Inferred 

Data 

Inferred 

Intercept  (BO) 

5.16 

3.57 

4.04 

5.85 

Slope  (Bl) 

1.12 

1.67 

1.53 

1.03 

Sigma 

0.183 

0.127 

0.107 

0.154 
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Figure  3.  d  versus  a  for  Ti-6246  Bolt  Hole  Specimens 
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Figure  6.  Threshold  Plot  Comparing  Results  from  Data  and  Use  of  the  Difference  Methods 
for  IN  718  Bolt  Hole  Specimens 


Figure  7.  Threshold  Plot  Comparing  Results  from  Data  and  Use  of  the  Difference  Methods 
for  Ti-6246  Bolt  Hole  Specimens 
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Figure  8.  Threshold  Plot  Comparing  Results  from  Data  and  Use  of  the  Ratio  Methods  for 
IN  718  Bolt  Hole  Specimens 


Figure  9.  Threshold  Plot  Comparing  Results  from  Data  and  Ratio  Methods  for  Ti-6246  Bolt 
Hole  Specimens 


Section  5 

Recommendations 


As  can  be  seen  in  the  preceding  analysis,  the  material  evaluated  in  this  report  will  not  provide  an 
adequate  measure  of  POD  capability.  The  alternatives  from  which  the  government  must  choose 
are  therefore  limited.  The  current  approach  to  this  problem,  in  effect,  is  to  ignore  material 
differences.  When  establishing  the  POD  for  a  material/geometry  combination  that  has  no 
associated  specimen  set,  the  procedure  is  to  use  the  specimen  set  of  the  same  geometry  of  a 
different  material.  The  POD  is  then  assumed  to  be  equal  to  that  found  from  this  specimen  set, 
ignoring  the  difference  in  material.  Veridian’s  report  concludes  by  this  quote:  “In  conclusion,  the 
best  possible  prediction  of  POD  is  simply  to  match  a  similar  geometry  and  use  that  data”  [2]. 

The  only  other  option  for  the  Air  Force  is  to  conduct  a  study  that  will  seek  to  establish  a  physical 
or  empirical  model  of  the  eddy  current  response.  The  goal  would  be  to  account  for  the  myriad 
variables,  or  show  that  any  given  variable  does  not  appreciably  affect  the  eddy  current  response. 
This  approach  would  predict  the  POD  from  known  responses  on  other  material/ 
geometry  combinations.  The  models  evaluated  in  this  report  are  effectively  models  without  any 
physical  or  empirical  bases.  They  are  completely  arbitrary  and,  in  some  ways,  do  not  make  sense 
physically.  Unless  and  until  a  physical  or  empirical  model  is  established  for  the  eddy  current 
response  for  these  material/geometry  combinations,  the  best  approach  seems  to  be  to  ignore 
material  differences. 
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Section  6 

Summary 


The  analysis  of  the  data  from  the  four  specimen  sets  showed  that  the  two  proposed  material 
correlation  methods,  the  difference  method  and  the  ratio  method,  did  not  predict  the  d  versus  a 
parameters  for  the  unknown  specimen  set  with  sufficient  accuracy  to  justify  their  use.  The 
correlation  models  were  not  based  on  physical  considerations  of  the  eddy  current  interactions  with 
the  material  or  geometry.  Further  complicating  successful  material  correlation  attempts  is  the 
difficulty  in  controlling  all  of  the  necessary  eddy  current  inspection  parameters.  Many  complicated 
material  correlation  reliability  tests  would  have  to  be  conducted  using  hybrid  scan  plans  and 
probes,  and  an  assortment  of  untested  assumptions  would  have  to  be  made.  Even  so,  the  results 
of  such  tests  have  been  shown  to  be  inadequate.  The  data  strongly  suggest  that  it  is  better  to  ignore 
material  differences.  The  other  option  is  to  conduct  a  study  establishing  a  physical  or  empirical 
model  of  the  eddy  current  response.  The  goal  would  be  to  account  for  the  myriad  variables,  or 
show  that  any  given  variable  does  not  appreciably  affect  the  eddy  current  response.  This  approach 
would  predict  the  POD  from  known  responses  on  other  material/geometry  combinations. 
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